Transformation-Induced Bias: Unbiased Coe icients Do Not Imply Unbiased Quantities of Interest

Political scientists commonly focus on quantities of interest computed from model coe icients rather than on the coe icients themselves. However, the quantities of interest, such as predicted probabilities, first di erences, and marginal e ects, do not necessarily inherit the small-sample properties of the coe icient estimates. Indeed, unbiased coe icient estimates are neither necessary nor su icient for unbiased estimates of the quantities of interest. I characterize this transformation-induced bias, calculate an approximation, illustrate its importance with two simulation studies, and discuss its relevance to methodological research. Political scientists use a wide range of statistical models yi ∼ f (θi ), where i ∈ {1, . . . ,N } and f represents a probability distribution. The parameter θi is connected to a design matrix X of k explanatory variables and a column of ones by a link function g , so that g (θi ) = Xi β . In the binary logit, for example, f represents the Bernoulli probability mass function and g represents the logit function, so that yi ∼ Bernoulli(πi ) and πi = logit−1(Xi β ). The researcher usually estimates β with maximum likelihood (ML), and, depending on the choiceofg and f , the estimate β̂ might havedesirable small-sampleproperties.However,MLdoes not produce unbiased estimates in general. For this reason,methodologists frequently use Monte Carlo simulations to assess the small-sampleproperties of estimators andprovideuserswith rules of thumb about appropriate sample sizes. For example, the ML estimates of β for the binary logit are biased away from zero, leading Long (1997, p. 54) to suggest that “it is risky to use ML with samples smaller than 100, while samples larger than 500 seem adequate.” Although methodologists tend to focus on estimating model coe icients, substantive researchers tend to focus on some other quantity of interest. A quantity of interest is simply a transformation τ of the model coe icients. Examples include marginal e ects, first and second di erences, predicted probabilities and expected values, and risk ratios (King, Tomz, and Wittenberg 2000). Fortunately, the invariance principle allows the researcher to calculate estimates of the quantities of interest from the coe icient estimates in a principled manner. The invariance principle states that if β̂ is the ML estimate of β , then for any function τ , the ML estimate of τ(β ) is τ(β̂ ) (King 1998, pp. 75–76, and Casella and Berger 2002, pp. 320–321). That is, researchers can simply transform the ML estimates of the model coe icients to obtain an ML estimate of the quantity of interest. Of course, if β̂ is a consistent estimator of β , then τ(β̂ ) must be a consistent estimator of τ(β ). But the invariance principle raises an important question: Does τ(β̂ ) inherit the small-sample properties of β̂ , such as unbiasedness or approximate unbiasedness? The answer is no; the estimates of the quantities of interest do not inherit the small-sample properties of the coe icient estimates. For example, a sample size of N = 250 that produces nearly unbiased coe icient estimates for a probitmodel can lead to bias in themarginal e ect estimates of 25%or more. Author’s note: All computer codenecessary for replication is available at https://github.com/carlislerainey/transformationinduced-bais and dx.doi.org/10.7910/DVN/CYXFB8 (Rainey 2017).